Bobby Tables but with LLM Apps - Google NotebookML Data Exfiltration · Embrace The Red

🌈 Abstract

The article discusses a security vulnerability in Google's NotebookML, an experimental project that allows users to upload files and analyze them with a large language model (LLM). The vulnerability is related to Prompt Injection, which can allow uploaded files to manipulate the chat conversation and control what the user sees in responses. Additionally, the article highlights that NotebookML is also vulnerable to data exfiltration when processing untrusted data.

🙋 Q&A

[01] Bobby Tables but with LLM Apps - Google NotebookML Data Exfiltration

1. What is Google's NotebookML?

• Google's NotebookML is an experimental project that allows users to upload files and analyze them with a large language model (LLM).

2. What vulnerability does NotebookML have?

• NotebookML is vulnerable to Prompt Injection, meaning that uploaded files can manipulate the chat conversation and control what the user sees in responses.
• There is currently no known solution to these kinds of attacks, so users can't implicitly trust responses from large language model applications when untrusted data is involved.
• NotebookML is also vulnerable to data exfiltration when processing untrusted data.

3. What can an attacker do with the Prompt Injection vulnerability?

• An attacker can instruct NotebookML to automatically render hyperlinks and images, which can be used as a data exfiltration channel.
• Besides displaying incorrect information to the user (e.g., scamming, etc.) during a prompt injection attack, an attacker can also gain access to other users' information (e.g., in the same document or other documents).

4. What is the demo setup and what does it demonstrate?

• A demo document with a proof-of-concept exploit was created, where the attacker's payload is injected into the "Sarah" record's description field.
• When the data is analyzed by NotebookML, the injected payload renders an image that sends the attacker's email and code to a third-party server.
• The attack is not limited to data inside the same document, as the attacker can also read data from a second document and append it to the exfiltration URL.

5. What is the severity of this vulnerability?

• The vulnerability is considered a high-severity security issue, as an attacker could read data from other documents and also add or remove rows from the output.

6. What was the responsible disclosure process?

• The vulnerability was responsibly disclosed to Google on December 4, 2023, but no mitigation has been put in place so far.
• After 132 days, the report is being made public to follow responsible disclosure industry norms for vulnerabilities that are not fixed in a reasonable time.

7. What are the recommended mitigations?

• Not rendering any images that are pointing to arbitrary domains.
• Not rendering any clickable hyperlinks to arbitrary domains.
• Users of NotebookML should be aware of the risks and avoid processing sensitive information or data from untrusted sources.